-
Notifications
You must be signed in to change notification settings - Fork 253
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix negative MV diff rate calculation when allow_high_precision_mv is false #3003
base: master
Are you sure you want to change the base?
Conversation
Codecov ReportBase: 86.52% // Head: 86.48% // Decreases project coverage by
Additional details and impacted files@@ Coverage Diff @@
## master #3003 +/- ##
==========================================
- Coverage 86.52% 86.48% -0.04%
==========================================
Files 89 89
Lines 34046 34046
==========================================
- Hits 29457 29445 -12
- Misses 4589 4601 +12
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here. ☔ View full report at Codecov. |
The logic is sound. I started an AWCY run on objective-1-fast at default speed:
|
I think the confounding factor is that this function is called for motion estimation on subsampled frames. |
@barrbrain I also saw your other commit (barrbrain@5b45ccf) At least for me, it wasn't obvious at first that it changes the rate curve itself rather than just the rounding (see below) https://www.desmos.com/calculator/ekdizwd572 perhaps the rate curve could somehow be adjusted for better gains? I'll look into it further |
This was a mistake, I shouldn't have changed the scale. I tried another experiment which gave modest gains on top of the rounding change, barrbrain@7baa93f :
However, the more surprising result was that if that change were applied first, the rounding change had no impact on decisions. Compared to the baseline, results for this experiment were a wash:
|
Hmm, I'll mark this as draft for now then unless I can find an improvement later on |
Bit shifting for division by a power of 2 causes incorrect rounding for negative numbers. This should very slightly increase quality as a result of more accurate rate distortion calculations.